A Comprehensive Survey on Applications of Transformers for Deep Learning Tasks

Transformer is a deep neural network that employs a self-attention mechanism to comprehend the contextual relationships within sequential data. Unlike conventional neural networks or updated versions of Recurrent Neural Networks (RNNs) such as Long Short-Term Memory (LSTM), transformer models excel in handling long dependencies between input sequence elements and enable parallel processing. As a result, transformer-based models have attracted substantial interest among researchers in the field of artificial intelligence. This can be attributed to their immense potential and remarkable achievements, not only in Natural Language Processing (NLP) tasks but also in a wide range of domains, including computer vision, audio and speech processing, healthcare, and the Internet of Things (IoT). Although several survey papers have been published highlighting the transformer's contributions in specific fields, architectural differences, or performance evaluations, there is still a significant absence of a comprehensive survey paper encompassing its major applications across various domains. Therefore, we undertook the task of filling this gap by conducting an extensive survey of proposed transformer models from 2017 to 2022. Our survey encompasses the identification of the top five application domains for transformer-based models, namely: NLP, Computer Vision, Multi-Modality, Audio and Speech Processing, and Signal Processing. We analyze the impact of highly influential transformer-based models in these domains and subsequently classify them based on their respective tasks using a proposed taxonomy. Our aim is to shed light on the existing potential and future possibilities of transformers for enthusiastic researchers, thus contributing to the broader understanding of this groundbreaking technology

Early satisfactory results of percutaneous repair in neglected achilles tendon rupture

Abstract Purpose This investigation aimed to study the outcome of percutaneous repair of Achilles tendon ruptures regarding patient-reported and objective outcomes. Methods This is a retrospective review of a cohort of patients (n = 24) who underwent percutaneous repair of neglected Achilles rupture in the period between 2013 and 2019. Included patients were adults with closed injuries, presented 4–10 weeks after rupture, with intact deep sensation. All underwent clinical examination, X-rays to exclude bony injury and MRI for diagnosis confirmation. All underwent percutaneous repair by the same surgeon, using the same technique and rehabilitation protocol. The postoperative assessment was done subjectively using ATRS and AOFAS score and objectively using a percentage of heel rise comparison to the normal side and calf circumference difference. Results The mean follow-up period was 14.85 months ± 3 months. Average AOFAS scores at 6,12 months were 91 and 96, respectively, showing statistically significant improvement from pre-op level (P < 0.001). Percentage of heel rise on the affected side and calf circumference showed statistically significant improvement over the 12 month follow up period (P < 0.001). Superficial infection was reported in two patients (8.3%), and two cases reported transient sural nerve neuritis. Conclusion Percutaneous repair of neglected Achilles rupture using the index technique proved a satisfactory patient-reported and objective measurement at a one-year follow-up. With only minor transient complications